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Introduction 

The principal ideas of harmonic analysis on a locally compact group G which 
is not necessarily compact or commutative were developed in the 1940s and early 
1950s. In this theory, the role of the classical fundamental harmonics is played 
by the irreducible unitary representations of G. The set of all equivalence classes 
of such representations is denoted by G and is called the dual object of G or the 
unitary dual of G. 

Since the 1940s, an intensive study of the foundations of harmonic analysis on 
complex and real reductive groups has been in progress (for a definition of reductive 
groups, the reader may consult the appendix at the end of §2). The motivation for 
this development came from mathematical physics, differential equations, differen- 
tial geometry, number theory, etc. Through the 1960s, progress in the direction of 
the Plancherel formula for real reductive groups was great, due mainly to Harish- 
Chandra's monumental work, while at the same time, the unitary duals of only a 
few groups had been parametrized. 

With Mautner's work [Ma], a study of harmonic analysis on reductive groups 
over other locally compact nondiscrete fields was started. We shall first describe 
such fields. In the sequel, a locally compact nondiscrete field will be called a local 
field. 

If we have a nondiscrete absolute value on the field Q of rational numbers, then 
it is equivalent either to the standard absolute value (and the completion is the 
field M of real numbers) or to a p-adic absolute value for some prime number p. 
For r e write r = p'^a/h where a, a, and b are integers and neither a nor h are 
divisible by p. Then the p-adic absolute value of r is 

A completion of Q with respect to the p-adic absolute value is denoted by Qp. It is 
called a field of p-adic numbers. Each finite-dimensional extension F of Qp has a 
natural topology of a vector space over Qp. With this topology, F becomes a local 
field. The topology of F can also be introduced with an absolute value which is 
denoted by | \f (in §5 we shall fix a natural absolute value). The fields of real and 
complex numbers, together with the finite extensions of p-adic numbers, exhaust 
all local fields of characteristic zero up to isomorphisms [We] . 

Let F be a finite field. Denote by F((X)) the field of formal power series over 
F. Elements of this field are series of the form / = On-^", a„ S F, for some 

©1993 American Mathematical Society 
0273-0979/93 $1.00 -I- $.25 per page 



1 



2 



MARKO TADIC 



integer k. Fix q > 1. Very often q is taken to be the cardinal number of the finite 
field F. One defines an absolute value of / by the formula 

\J\¥{{X)) -q ^ ' -r 

when / 7^ 0. In this way ¥{{X)) becomes a local field. Fields of formal power series 
over finite fields exhaust all local fields of positive characteristic up to isomorphisms 
[We]. 

The fields M and C arc called archimedean fields. For any x,y £ or x,y € , 
there is always a positive integer n such that |y| < \nx\. The above property does 
not hold for any other local field. This is the reason that local fields which are not 
isomorphic to K or C are called nonarchimedean local fields. 

After Mautner in the 1960s, a series of people started to consider reductive groups 
over nonarchimedean local fields. Let us recall that p-adic fields were introduced 
historically to enable one to consider a single equation over a p-adic field instead 
of an infinite series of congruences mod p''. Arithmetical problems also provided 
motivation to consider representations of reductive groups over such fields. The 
strongest motivation comes from the Langlands program. A unifying element in 
this program is the representation theory of reductive groups. A nice introduction 
to the Langlands program is [Gb3] . 

Let G be a reductive group over a local field. Harish-Chandra created a strat- 
egy for obtaining the unitary dual G through the nonunitary dual G, where G is 
the set of all functional equivalence classes of topologically completely irreducible 
continuous representations of G. Functional equivalence means that the matrix 
coefficients of one representation may be approximated by matrix coefficients of 
another on compact sets, and vice versa. A complete definition of G is in §2. To 
obtain G, one needs to classify G (the problem of the nonunitary dual) and to 
identify G C G (the unitarizability problem). In [L2] Langlands showed how to 
parametrize G by irreducible representations with certain good asymptotic prop- 
erties (tempered representations) of reductive subgroups, when the field F is M. 
The tempered representations were classified for F = R by Knapp and Zuckerman 
[KnZu] on the basis of Harish-Chandra's work, thus providing a complete picture 
of G. Despite the Langlands classification of G in the archimedean case, there were 
no big breakthroughs in the classification of unitary duals for quite a long time. 
Borel-Wallach and Silberger proved that Langlands parametrization of G in terms 
of tempered representations of reductive subgroups was valid for reductive groups 
over all local fields [BlWh, Sil]. 

In this paper, we shall be concerned with the unitarizability problem for reductive 
groups over local fields. One usually breaks the unitarizability problem into two 
parts. The first part is constructing elements of G, and the second is showing 
that the constructed representations exhaust G (completeness argument). The 
completeness argument is usually realized by showing that the classes of G\G arc 
not unitarizable. We may call such an approach to the completeness argument 
indirect. 

Suppose that the field is archimedean. Then one can linearize the problems for G 
(and G); one can "diflFerentiate" the representations and come to the infinitesimal 
theory where the main object is the Lie algebra g of G. Also, for a maximal compact 
subgroup K of G, the theory of compact Lie groups gives an explicit description 
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of K. Thus, one may try to understand (tt, H) e G by studying the restriction of 
TT to K. These two points explain why, in the archimedean case, some problems 
concerning representations, and especially the unitarizability problem, were often 
approached by studying the internal structure of representations. Let us recall that 
the internal approach was very successful in the compact Lie group case (restriction 
to a maximal torus). In the nonarchimedean case, there is no possibility of such an 
internal approach to the unitarizability problem. One of the reasons that there has 
been much less study of the unitarizability problem is that the nonunitary duals 
are not yet completely parametrized there. 

Despite the fact that unitary duals of a very restricted number of groups have 
been classified, it is interesting to note that in 1950 Gelfand and Naimark published 
a book [GfN2] in which they constructed what they assumed to be the dual ob- 
jects of the complex classical simple Lie groups. Their lists were very simple, and 
the representations were also simple (although infinite dimensional). Gelfand and 
Naimark were using functional analytic methods as tools in their analysis. In 1967 
Stein constructed, in a fairly simple manner, representations in GL(2n, C)' which 
were not contained in the lists of Gelfand and Naimark [St] . For some other classical 
groups, it was even easier to see the incompleteness of the lists from [GfN2] . 

The representations of Gelfand and Naimark of GL(n, C), complemented by 
Stein, were not generally expected to exhaust the whole of GL(n, C)". 

The main aim of this paper is to present the ideas which lead first to the solution 
of the unitarizability problem for GL(n) over nonarchimedean local fields [Td3] and 
to the recognition that the same result holds over archimedean local fields [Td2], 
a result which was proved by Vogan [Vo3] using an internal approach. Let us say 
that the approach that we are going to present may be characterized as external. 
At no point do we go into the internal structure of representations. 

Let us present the answer. We fix a general local field F. Let be the set 
of all functional equivalence classes of irreducible square integrable modulo center 
representations of all GL{n, F),n > 1 (for definition see §3). Let | |f be the 
modulus of F (see §5). For each representation 5 G of GL(n,i^), and for each 
m > 1, consider the representation of GL{mn, F) parabolically induced (§1) by 

i det fr'^/'S ® I det 1^"-')/'^ ® • • • ® I det \-^^-'^/'6 

from a suitable standard parabolic subgroup (i.e., from one containing the upper tri- 
angular matrices, §2). This representation has a unique irreducible quotient which 
will be denoted by u{S, to). For < a < 1/2 let Tr{u{S, m), a) be the representation 
of GL(2TOn, F) parabohcally induced by 

I det \pu{S, to) ® I det \p"u{S, m). 

Denote by B all possible u{d,m) and Tr{u{S,m),a). Then the answer is 

Theorem, (i) Let n, . . . ,t„ e i?. Then the representation tt parabolically induced 
by 

n (g) • • • (g) r„ 

of suitable GL(to, F) is irreducible and unitary. 

(ii) Suppose that p is obtained from ai,.. . ,a„ € B in the same manner as tt 
was obtained from n , . . . , in (i) . Then n = p if and only if n = m and the 
sequences (n, . . . , r„) and (ci, . . . , cr„) coincide after a renumeration. 
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(iii) Each irreducible unitary representation of GL{m,F), for any m, can be 
obtained as in (i). 

A new, and at the same time very old, point of view that led to the papers [Tdl- 
Td3] was that the unit ariz ability problem has a reasonably simple answer and that 
the unitary representations appear in simple and natural ways. In the completeness 
argument, instead of an indirect strategy, a direct argument was used. In this 
way, a detailed study of nonunitary representations was avoided. In all arguments, 
essentially only Hilbert space representations were necessary. 

Surprisingly, the statement of the theorem, which was first discovered in the 
nonarchimedean case in [Td3], says that in the case F = C the unitary dual of 
GL(n,C) should consist of the representations of Gelfand, Naimark, and Stein. 
Not only the statement of the nonarchimedean case of the theorem, but also the 
methods of the proof in [Td3] made sense in the archimedean case. Thus, after a 
complete proof had been written in the nonarchimedean case, we wrote in [Td2] 
the proof of the archimedean case of the theorem. We have used there a theorem 
of Kirillov from [Kil], for which he never published the complete proof (see §9). 

There are now many solutions of the unitarizability problems, especially for 
particular reductive groups in the archimedean case. In general, they are based on 
ideas different than the one that we present in this paper. Two of them take a 
distinguished place (they solve completely the problem for a series of groups having 
no bounds on their semisimple split ranks) . The first is Vogan classification in [Vo3] 
of the unitary duals of GL(n) over R, C, and H. He proved a theorem equivalent 
to the statement of our theorem for archimedean F. The other one is Barbasch's 
classification of the unitary duals of the complex classical Lie groups in [Bb] . 

Since we consider both the archimedean and nonarchimedean cases, it is natural 
to recall Harish-Chandra's Lefschetz principle: "Whatever is true for real reduc- 
tive groups is also true for p-adic groups" [IIa2]. One problem with the Lefschetz 
principle is that we usually obtain a result for archimedean F by one kind of con- 
siderations and for nonarchimedean F by very different methods, and after that 
we compare the results. An interesting problem is to explain the phenomenon of 
the Lefschetz principle, which is certainly related to our depth of understanding of 
these two theories. An important task of this paper is to present a unified point 
of view on the theorem and its proof. We shall discuss both the theorem and the 
proof, making no distinction concerning the nature of the field. This is possible by 
the use of the external point of view. In our approach we shall very often be close 
to the point of view of the representation theory of general locally compact groups, 
and this will be a unifying point. 

Professor P. J. Sally suggested that I write a paper where the ideas would be 
extracted from the technical machinery as much as possible. I am thankful to 
him for his advice and encouragement and for his generous help during the writing 
of various drafts of this paper, which he has read. I am also thankful for the 
hospitality and support of the University of Chicago during the shimmer of 1986 
when a sketch of this paper was prepared. I would like to express my thanks to 
a number of mathematicians for helpful discussions or encouragement during the 
writing of this paper and the preceding ones on the same topic. Let me mention 
just a few of them: M. Duflo, I. M. Gelfand, H. Kraljevic, D. Milicic, F. Rodier, 
P. J. Sally, and M. F. Vigneras. H. Kraljevic gave very helpful comments on the 
first draft of this paper. Discussions with D. Milicic and his suggestions were of 
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great help in the preparation of the paper. The referee gave a number of valuable 
and helpful suggestions to the preceding draft of this paper. Among others, the 
work of J. Bernstein and A. V. Zelevinsky greatly influenced the development of 
some ideas presented in this paper. The final step in preparation for publishing 
this paper was done in Gottingen. I am thankful for hospitality and support to the 
Sonderforschungsbereich 170 and S. J. Patterson. 

We hope that this paper will illustrate a certain internal symmetry in the external 
approach to the unitarizability problem. We hope that some of our ideas will be 
helpful in dealing with the unitary duals of other nonarchimedean classical groups 
(and also nonunitary duals). The papers [SlTd] and [Td8] indicate that this hope 
is not without basis. 

Finally, we introduce some general notation which we shall use throughout the 
paper. For a topological space X, C{X) will denote the space of all continuous 
(complex- valued) functions on X. The subspace of all compactly supported con- 
tinuous functions will be denoted by Cc{X). If we have a measure jj, on X, then 
L^(X, /i) will denote the space of all classes of ^-integrable functions on X and 
L"^ {X, jj) will denote the Hilbert space of all classes of square integrable functions 
on X with respect to /x. For a smooth manifold X the space of smooth functions 
on X wiU be denoted by C°^{X) and C°°(X) n Cc(X) will be denoted by C^{X). 
If X is a totally disconnected locally compact topological space, then {X) will 
denote the space of all compactly supported locally constant functions on X. The 
fields of real and complex numbers are denoted by K and C respectively. The ring 
of integers is denoted by Z, nonnegative integers are denoted by Z+, and positive 
ones are denoted by N. 

1. Concept of harmonic analysis 
on general locally compact groups 

In this section we shall outline some of the ideas of harmonic analysis on locally 
compact groups. 

Let G be a locally compact group. We shall always suppose in this paper that 
the groups are separable. A representation of G is a pair (tt, V) where y is a com- 
plex vector space which is not zero dimensional and tt is a homomorphism of G 
into the group of all linear isomorphisms of V . By a continuous representation of 
G we shall mean a representation (tt, H) of G where H \s a Hilbert space and the 
map {g, v) i— > TT{g)v, G x i? — > iJ is continuous (we shall always assume that H is 
separable). A closed subspace H' of H will be called a subrepresentation of a con- 
tinuous representation {tt,H) of G, if H' is invariant for all operators TT{g), g € G. 
A continuous representation (tt, H) of G is called irreducible if there does not exist a 
nontrivial subrepresentation of (tt, H) (i.e., different from {0} and H). A continuous 
representation (tt, H) of G will be called a unitary representation if all the operators 
'^{9)^9 G G, are unitary. Two unitary representations {'jTi,Hi),i = 1,2, of G are 
called unitarily equivalent if there exists a Hilbert space isomorphism (p : Hi H2 
such that n2{g)f = <fTri{g) for all g £ G. The set of all unitarily equivalence classes 
of irreducible unitary representations of G will be denoted by G and called the 
unitary dual of G. For a family (tt, , i?,), ?' G /, of luiitary representations of G, 
there is a natural unitary representation of G on the direct sum of Hilbert spaces 
0.gj Hi. This representation will be denoted by (0^^/ tt^, 0jg/ Hi). It is called a 
direct sum of representations [iTi^Hi), i e /. 
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A continuous representation (resp. unitary representation) on a one-dimensional 
space is called a character (resp. unitary character) of G. 

The main problem of harmonic analysis on the group G is to understand some 
interesting unitary representations of G (such representations are usually given on 
function spaces). One way to study this problem is to break it into two parts (at 
least for type I groups which will be described in the sequel of this section and which 
will be the only groups considered in this paper). The first part is to understand 
irreducible unitary representations, i.e.. G, and the second part is to understand 
other unitary representations in terms of irreducible ones. This strategy is in the 
spirit of Fourier's classical idea of fundamental harmonics. 

Let us explain what we mean by understanding general unitary representations 
in terms of irreducible ones. If G is a compact group, then a fundamental fact 
is that each unitary representation of G can be decomposed into a direct sum of 
irreducible miitary representations [Di. Theorem 15.1.3.]. Understanding a miitary 
representation tt in terms of irreducible unitary representations means, in the com- 
pact case, to know how to decompose tt into a direct sum of irreducible unitary 
representations. In the noncompact case each imitary representation decomposes 
into a direct integral of irreducible unitary representations, and understanding here 
again means to know how to decompose a given unitary representation into a direct 
integral of irreducible representations. We are not going to define here the notion 
of direct integral of representations because the definition is quite technical. The 
interested reader may consult [Di, §8] . Let me only mention that the direct integral 
generalizes the notion of direct sum and that direct integrals are determined by 
measures on G. To consider measures on G, one needs some cr-ring of sets. This 
(T-ring arises in a standard way from the natural topological structure on G. Now 
we shall define this topology. 

If (tt, H) is a continuous representation of G and v,w € H, then the function 
g H- > {'K{g)v,w) on G is called a matrix coefficient of (tt,!!). We denote by $(7r) 
the linear span of all matrix coefficients of (tt, H). The closure operator on subsets 
of G is defined as follows. Let n G G and X C G. Then tt G Cl{X) if and only if 
each element of ^{ir) can be approximated uniformly on each compact subset of G 
by elements from IJaex ^ (''')• This topology on G will be called the topology of the 
unitary dual of G. 

If G is a commutative group, then each irreducible unitary representation of G 
is given on a one-dimensional space (this follows easily from the spectral theorem). 
Thus each tt G G is a function. Pointwise multiplication of element of G defines 
a group structure on G. Together with the above topology, G is again a locally 
compact commutative group which is called the dual group of G. The role of the 
topology of the unitary dual is crucial in harmonic analysis on locally compact 
commutative groups. This topology is a basis on which one builds the fundamental 
facts of the harmonic analysis on commutative groups. One of these fundamental 
facts is that G and {G'Y are canonically isomorphic (Pontryagin duality). 

In the case of noncommutative groups, the role of this topology on G is less crucial 
than in the commutative case, but it is still an important and natural object to 
consider. For example, G is a type I group if and only if G is a To-space, i.e., for 
any two different points in G, at least one of them has a neighborhood which does 
not contain the other one [Di, 9.5.2. and Theorem 9.1]. It is important to notice 
that, in general, G is not topologically homogeneous and there exist significant 
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connections between properties of irreducible representations and their position in 
G with respect to the topology. 

Before we proceed further, we shall say a few words about some measures which 
arc natural to consider on locally compact groups. For a locally compact group G, 
there exists a positive measure invariant for right translations. Such a measure will 
be called a right Haar measure on G and it will be denoted by hg- Thus, 



for any / G Cc (G) and x G G. Any two right Haar measures are proportional. The 
behavior of a right Haar measure for left translations is described by the modular 
function. There exists a function Aq on G such that 



for any / G Cc{G) and x G G. The group G is called unimodular if Aq = 1, 
i.e., if hg is also invariant for left translations. For more information about Haar 
measures and for proofs of the above facts one may consult [Bu2] . 

Suppose that G is unimodular. The space Cc{G) becomes an algebra for the 
convolution which is defined by the formula 



/i,/2 G Cc{G). In a natural way one can extend the convolution to L^{G,^ig)- 
Then L^{G,ij.g) becomes a Banach algebra. For / G Cc(G) and a continuous 
representation tt of G set 



Now TT becomes a representation of the convolution algebra Gc(G), and this repre- 
sentation is called the integrated form of the representation tt of G. If tt is unitary, 
then the last formula also defines a representation of the algebra L^{G, hg)- More- 
over, it is a ^-representation if we define f*{g) = f{g^^)- 

We have already mentioned that the basic problem of harmonic analysis on 
G is to classify G and then to decompose interesting representations in terms of 
G. Among the interesting representations, there is one that should be the first 
to be understood, namely, the regular representation on L'^{G, hg)- By the ab- 
stract Planchcrcl theorem, for a unimodular group G there exists a unique positive 
measure u on G such that 



for ah / G L^{G,ijlg) ^L^{G,ijlg) pi, Theorem 18.8.2]. The measure v is called 
the Plancherel measure of G (it determines an explicit decomposition of L^{G, ijlg) 
into a direct integral of elements of G). 
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While the basic ideas of harmonic analysis on general locally compact groups 
were laid down in the 1940s, one of the first breakthroughs in classifying unitary 
duals was the work of Kirillov for nilpotent Lie groups at the beginning of the 1960s 
(see [Ki2, §§13 and 15]). 

Let us first recall that a Lie group is a group supplied with a structure of a (real) 
analytic manifold such that the group operations (i.e., multiplication and inversion) 
are analytic mappings. We can define the Lie algebra g of a Lie group G as the 
tangent space of G at the identity, supplied with a bracket operation [ , ] which can 
be defined in the following way. If X, F e are tangent vectors to curves x{t), y{t) 
for t = respectively, then [X, Y\ is the tangent vector to the curve 

t ^ x{T)y{T)x{T)~'^y{T)~'^ 

where r = sgn (t)|<|-^/^, a,t t = [Ki2, 6.3.]. An element g & G acts on G by inner 
automorphism. The differential of this action is denoted by Ad{g). In this way G 
acts on g, and this action is called the adjoint action of G on g. 

Let G be a connected simply connected nilpotent Lie group, and let g* be the 
space of linear forms on the Lie algebra g of G. There is a natural action of G on g*. 
It is called the coadjoint action of G. Using the theory of induced representations 
(which we shall discuss a bit later in this section), Kirillov established a canonical 
one-to-one mapping from the set of all coadjoint orbits onto the unitary dual of G 

G\g* ^ G 

which gives a simple description of the unitary dual. With a natural topology on the 
left-hand side, this is a homeomorphism. Kirillov theory also gives the characters 
of irreducible unitary representations and the Plancherel formula for nilpotent Lie 
groups. 

In the second part of this section, we define some notions that we shall need in 

the sequel. 

If G is a compact group, then each tt G G is given on a finite-dimensional space 
[Di, 15.1.4.]. As in the theory of finite group representations, the function 

: gi-^ Trace 7r(g), 

which is called the character of the representation tt, completely determines the 
class of TT in G. We have a right Haar measure on G. Thus the character function 
determines a distribution on a compact Lie group G. It is easy to see that this 
distribution is 

/^Ttace7r(/),/GGr(G), 

which will be denoted by again. If tt is an infinite-dimensional representation, 
obviously the trace as a function is not well defined. Nevertheless, it may happen 
that the above distribution is well defined. For an arbitrary Lie group G, one can 
take the above distribution for the definition of the character of tt G G, if 7r(/) is 
trace-class for / e G^(G). If 7r(/) has a trace for / e G^(G), then 7r(/) must be a 
compact operator for any / G L^{G, /ic)- If all tt G G have characters in the above 
sense, then n{f ) is a compact operator for any tt G G and / G L^{G,iig)- 

A locally compact group is called a CCR-group if n{f) is a compact operator for 
any tt G G and / G L^{G, hg)- All CCR-groups are of type I [Di, Proposition 4.3.4. 
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and Theorem 9.1]. A great number of very important groups are CCR-groups. The 
most important classes of CCR-groups are commutative groups, compact groups, 
nilpotent Lie groups, and reductive groups over local fields (reductive groups will be 
defined in the appendix at the end of the following section). In particular, classical 
groups over local fields are CCR-groups. One can characterize CCR-groups in terms 
of the topology of the unitary duals. A group G is a CCR-group if and only if G is 
a Ti-space, i.e., if the points arc closed subsets of G [Di, 9.5.3.]. 

For some important classes of groups one can show that they are CCR-groups by 
showing that they have so-called "large" compact subgroups. Now we shall explain 
the last notion. 

Let G be a locally compact unimodular group, and let K he a compact subgroup. 
For d E K and a continuous representation (tt, i/^) of G, let Ht^{S) be the subspace 
of spanned by all subrepresentations of tt\K which are isomorphic to S (here 
Tr\K denotes the representation of K obtained by restriction from G). If 

dimci?7r(^) < oo 

for all S € K, then tt is called a representation of G with finite K- multiplicities. 
One calls K a large compact subgroup of G if for each 6 G K the function 

TT dime H^{5) 

is a bounded function on G. If G has a large compact subgroup, then G is a CCR- 
group [Di, Theorem 15.5.2]. Some very important classes of groups have large 
compact subgroups, for example, reductive groups over local fields. 

For a continuous representation (tt, H) of G, a vector v & H \s called K-finite if 
the span of all TT{k)v, k £ K, is finite dimensional. 

In the rest of this section we shall discuss some parts of the theory of the induced 
representations for locally compact groups. The notion of induced representations 
for locally compact groups generalizes the well-known notion of induced represen- 
tations for finite groups that was introduced and studied by Frobenius and Schur. 
Induction is one of the simplest and most important procedures for obtaining new 
representations of locally compact groups. 

The most important case of induction for the purpose of this paper is parabolic 
induction. To define this notion, it is enough to consider a closed subgroup P of 
a unimodular group G and assume that there exists a compact subgroup K of G 
such that PK = G. Let (cr, M) be a continuous representation of P. The space of 
all (classes of) measurable functions f : G ^ H which satisfy 

f{pg) = Ap(p)i/V(p)/(5), peP, geG, 

and 

m'= I \\f{kWd„K{k) < oo 

Jk 

will be denoted by Indp((T). It is a Hilbert space. The group G acts by right 
translations on Indp(c7). This action we denote by R. Thus 



{RJ){x) = f{xg). 
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With this action, Indp(o-) is a continuous representation of G. It is unitary if a is 
unitary. 

In our considerations G will be a reductive group over a local field, while P will 
be a parabolic subgroup of G (these terms will be defined in the following section). 
One will take a Levi decomposition P = MN of P and a continuous representation 
a of M. Since P/N = M, we shall consider u as a representation of P. Then we 
shall say that Indp(a) is a parabolically induced representation of G by o" . 

We shall also talk at some points in this paper about induced representations 
which are of more general type. Let G be a locally compact group which does 
not need to be unimodular, and let G be a closed subgroup of G. Suppose that a 
unitary representation (cr, H) of G is given (we may assume that ct is a continuous 
representation only). We denote by Ind^(c7) the space of all measurable functions 
f : G ^ H which satisfy 

f{cg) = [Acic)AG{g)-']'/Mc)f{g), g&G,ceC. 

One square integrability condition is required also. This condition is more technical 

than in the case of parabolic induction [Ki2, 13.2]. Again G acts by right transla- 
tions on Ind^(c7). This is a unitary representation of G. We say that Ind§(cr) is 
unitarily induced by a. 

Mackey obtained a simple criterion for testing if a given unitary representation 
of G is unitarily induced from G (Imprimitivity Theorem, [Ki2]). There is an im- 
portant specialization of this theory. If G contains a nontrivial normal abelian 
subgroup A'^, then Mackey theory implies a description of G by irreducible unitary 
representations of smaller groups (G and N need to satisfy certain general topo- 
logical conditions). Let X ^ ^ and denote by G^ the stabilizer of x in G {G acts 
on TV because G acts on N by automorphisms). Note that N consists of unitary 
characters. Take an irreducible unitary representation a of G^ such that a\N is a 
multiple of Then Ind^^ (a) is an irreducible unitary representation of G. One 
obtains all irreducible unitary representations of G by the above construction. Two 
such representations constructed from XijO"! and X2)f2 are equivalent if and only 
if Xij^i and X2,o'2 are conjugate. This specialization of Mackey theory is usually 
called small Mackey theory. In the special case when G is a semidirect product of 
a closed normal abelian subgroup N and a closed subgroup M, the small Mackey 
theory describes G more simply. Here G is parametrized by unitary characters of 
N and irreducible unitary representations of their stabilizers in M, divided by a 
natural equivalence. 

For our purpose, the case of semidirect products is the most interesting one. It is 
important to observe that one obtains here automatically the irreducibility of some 
induced representations. For a more detailed exposition of small Mackey theory, 
one may consult [Ki2, 13.3]. 

2. The nonunitary dual as a tool for the unitary dual 

In the study of the representation theory of the general linear groups, and more 
generally, of the classical groups, the terminology of the theory of reductive groups 
is very useful and natural. We shall very briefly recall some of the terminology of 
this theory in the first part of the appendix at the end of this section. The reader 
may also skip over the general definitions and follow only the case of GL(n) where 
we shall not need these general definitions. 
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We begin this section with a few definitions. We shall first define parabolic 
subgroups in GL(n, F). Take an ordered partition 



a = (ni,n2, ... ,nk) 



of n. Consider block-matrices 



■All 



A 



Ik 



A = 



-Afei Afefe- 

where Aij is an rii by nj matrix. Denote 

Pa = {A€ GL(n, F);Aij = for z > j}, 

M„ = {A e Gh{n,F);A,j = for i ^ j}. 

Let Na be the set of all A G Pa such that all An are identity matrices. Now Pa is 
called a standard parabolic subgroup of GL(n, F), Ma is called a Levi factor of P^, 
and Na is called the unipotent radical of Pa- The subgroup P(i,i,...,i) of all upper 
triangular matrices in GL(n, F) is called the standard minimal parabolic subgroup. 

Take any g € GL(n,F) and any ordered partition a = (ni,n2,... ,nk) of n. 
Then gPag~^ is called a parabolic subgroup of GL(n,F). Set P' = gPag~^, M' = 
gMag~^, and A''' = gN^g''^. Then P' = M'N' is called a Levi decomposition of P' . 
The group M' is called a Levi factor of P', and N' is called the unipotent radical 
of P' . Similarly, a minimal parabolic subgroup is defined to be any conjugate of 
the standard minimal parabolic subgroup. 

We shall now introduce the nonunitary dual. We denote by G a reductive group 
over a local field F. There exists a maximal compact subgroup K of G such that 
PminK = G for some minimal parabolic subgroup Pmin of G. We fix such a maximal 
compact subgroup. The Iwasawa decomposition PmmK = G holds for any maximal 
compact subgroup ii" of G if P is an archimedean field. If it is not, this may not be 
true for all maximal compact subgroups. The group K is & large compact subgroup 
of G in the sense of the previous section. In the case of GL(n, F) and P = R (rcsp. 
P = C), one may take for K the group of orthogonal matrices (resp. the group 
of unitary matrices). If P is nonarchimedean, one may take K in GL(n,P) to be 
GL(n, Op), where Op is the ring of integers in P, that is. Op = {a; G P; \x\f < !}• 
In a general linear group over any local field, all maximal compact subgroups are 
conjugate. 

We shall always assume in the sequel that continuous representations of G that 

we consider have finite multiplicities. 

For a continuous representation tt of G, we have denoted by $(7r) the linear span 
of all matrix coefficients of tt. This is a subspace of C(G). Denote by Cl'I>(7r) the 
closure of $(7r) with respect to the open-compact topology on C{G). Then two 
continuous representations tti and 772 of G are said to be functional equivalent if 



Cl$(7ri) = Cl$(7r2). 
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We denote by G the sot of all functional equivalence classes of continuous irreducible 
representations of G which have finite if- multiplicities. These representations are 
precisely the topologically completely irreducible representations of G (for a def- 
inition of the last notion, one may consult [Wr, 4.2.2.]). The set G is called the 
nonunitary dual of G. We could use Naimark equivalence to define G instead of 
the functional equivalence. We would get the same object. Two continuous repre- 
sentations (7ri,ifi) and {-it2,H2) are called Naimark equivalent if there exist dense 
subspaces Vi C Hi and V2 C H2 which are invariant for integrated forms and a 
closed one-to-one linear operator ip from Vi onto V2 such that 

for any / E Gc{G),x eVi. 

For an irreducible continuous representation tt of G which has finite if -multiplicities, 
the operator n{f) is of trace-class if / £ C^{G). The linear form 

/ ^ Trace(7r(/)) 

is denoted by and called the character of the representation ir. Two irreducible 
continuous representations with finite if -multiplicities are functionally equivalent 
if and only if they have the same characters. Furthermore, characters of represen- 
tations from G arc linearly independent. 

The natural mapping G — > G is one-to-one (G is a CCR-group). Therefore, we 
shall identify G with a subset of G. A class tt € G will be called unitarizable or 
unitary if tt G G C G. One supplies G with a topology in the same way as we have 
supplied G with the topology of the unitary dual. 

The reader may consult the second part of the appendix at the end of this section 
for standard realizations of the set G. Those realizations depend on whether the 
field is archimedean or not. 

The idea of Harish- Chandra was to break the problem of describing G into two 
parts: the problem of the nonunitary dual and the unitarizability problem. The 
problem of determining the nonunitary dual appeared to be much more manageable 
than the unitarizability problem. 

Besides the above general strategy, there could be other strategies for getting G. 
It would be interesting to obtain a classification of the unitary duals dealing with 
nonunitary representations as little as possible or perhaps not at all. A strategy 
of such classification for GL(n) over archimedean fields dealing with only unitary 
representations can be based on a paper from 1962 by Kirillov [Kil]. We shall 
return to [Kil] in §9. Now we shall outline the strategy. 

Let Pn be the subgroup of GL(n, F) of all matrices with the bottom row equal 
to (0,0,... ,0,1). An interesting property of P„ follows from the small Mackey 
theory: P„ is in a bijcction with 

GL(n - 1, FY U GL{n - 2, F)" U • • • U GL(2, Fy U GL(1, Fy U GL(0, Fy 

(GL(G, F) denotes the trivial group). Gelfand and Naimark showed alrciady in 1950 
that certain irreducible unitary representations of SL(n,C), which they expected 
to exhaust SL(n, C)', remain irreducible as representations of where P^ de- 
notes the subgroup of SL(n, C) of all matrices with the bottom row of the form 
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(0, . . . ,0, x), X e C*^ . They obtained this result from the explicit formulas for those 
representations. Kirillov's idea was to first find a general proof of the irreducibility 
of n\Pn for TT G GL(n, Py when F = M or C. Then one has an inductive procedure 
for classification. After classifying GL(to, Py for m < n — 1, one also has a classifi- 
cation of P„ by the above remark about P„. Thus, the second part of the strategy is 
to find all possible extensions of representations from P„ to unitary representations 
of GL(n, F). This strategy was used by I. J. Vahutinskii in his study of irreducible 
unitary representations of GL(3,IR). 

At the end of this section we shall say a few words about the characteristics 
of some approaches to the unitary duals of certain groups in two relatively simple 
cases. 

We shall first consider the case of a compact Lie group G. We shall assume that 
G is connected. One starts from an irreducible unitary representation (tt, H) of 
G, and studying the internal structure of H, one comes to exact parameters which 
classify G . Let us give a rough idea of how this approach goes. A closed connected 
commutative subgroup of G is called a torus in G. Each torus is isomorphic to some 
T" where T is the group of all complex numbers of norm one. We fix a maximal 
torus T in G. Then each element of G is conjugate to an element of T, i.e., each 
conjugacy class of G intersects T. Suppose that tt is a continuous representation 
of G on a finite-dimensional space H. One can choose an inner product on H 
invariant for the action of G. Thus the representation 7r|T decomposes into a direct 
sum of some unitary characters Xi) • • • > Xm- These unitary characters are called the 
weights of tt with respect to T. Take now (tt, H) S G. We have already mentioned 
that the character 6^ of the representation tt determines the class of tt. Since the 
character 9,ri as a function on G, is obviously constant on conjugacy classes, tt is 
already determined by tt\T, i.e., tt is determined by its weights. Further analysis of 
the structure of the representation tt on H requires the study of the representation 
of the Lie algebra (the formula (L.A.) in the appendix at the end of this section, 
defines the action of the Lie algebra). It gives that there is a particular weight 
among all weights of tt which already characterizes tt (the highest weight). In this 
way one obtains a parametrization of G by a certain subset of characters of T (the 
dominant weights). For an exposition of this nice theory of Wcyl and Cartan, one 
may consult, among many nice expositions, the seventh paragraph of [Bui]. 

We shall present now a simple strategy for solving the unitarizability problem for 
G = SL(2,R). One may take for K the group S0(2) of all two-by-two orthogonal 
matrices of determinant one. Note that SO (2) = T. The unitary dual of K is given 
by the characters 

cos{(p) — sin(93) 
sin((p) 003(95) 

when n runs over Z. If (tt, H) £ G, then it is not hard to show that multiplicities 
of the representation tt\K are one, i.e., H{a„) are either zero- or one-dimensional 
subspaces. This can be seen in a similar way as one shows that the space of K- 
invariant vectors in an irreducible representation is one dimensional if (G, K) is a 
Gelfand pair [GfGrPi, Chapter III, §3, no. 4]. Thus, there is a basis {vn;n G S} of 
the Hilbert space parametrized by a subset S of Z. Suppose that tt is a unitary 
representation with a G-invariant inner product ( , ) on H. The formula (L.A.) 
from the appendix defines the representation of the Lie algebra g of G on /^-finite 
vectors. Differentiating the relation {TT{g)v,TT{g)w) = {v,w),g € G, along one- 
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parameter subgroups, one gets that the representation on K-&mte vectors satisfies 



for all X € g. Put \\vn\\ = c„. Since v„,n G S, are orthogonal, the inner product 
is completely determined by numbers c„,n e 5. One can solve the unitarizability 
problem in the following way. Take (tt, H) S G and check if there exist positive 
numbers c„,n e S, such that the inner product 



satisfies {tt{X)v,iu) + {v,Tr{X)w) = for all ii'-finite vectors v and w in H and all 
X e fl. All TT for which there exist such numbers form the unitary dual. Clearly, 
to be able to solve the above problem, we need to know explicitly the internal 
structure of irreducible representations of G. 

In the above two examples, one solves the problem by study of the internal 
structure of representations. 



Algebraic groups. We shall recall very briefly in the first part of this appen- 
dix some definitions from the theory of algebraic groups. For precise definitions 
containing all details, one should consult [Bl]. 

A linear algebraic group G is a Zariski closed subgroup of some general linear 
group with entries from an algebraically closed field. A linear algebraic group is 
called unipotent if it is conjugate to a subgroup of the upper triangular unipotent 
matrices. A linear algebraic group G is called reductive if G does not contain a 
normal Zariski closed unipotent subgroup of positive dimension. A linear algebraic 
group G is called semisimple if it is reductive and if it has a finite center. If a 
noncommutative linear algebraic group G does not contain a proper normal Zariski 
closed subgroup of positive dimension, then G is called a simple algebraic group. If 
there is a group isomorphism of G onto some GL(1)" which is also an isomorphism 
of algebraic varieties, then G is called a torus. 

A Zariski closed subgroup P of G is called a parabolic subgroup if the homoge- 
neous space G/P is a projective variety. 

By a reductive group, we shall mean the group G(F) of all F-rational points of 
a reductive group G which is defined as an algebraic variety over a local field F. 
By a parabolic subgroup of a reductive group G{F), we shall mean the group of all 
rational points of a parabolic subgroup of G, which is defined over F. A minimal 
parabolic subgroup of G{F) is a parabolic subgroup which docs not contain any 
other parabolic subgroup. All minimal parabolic subgroups in G{F) arc conjugate. 
If we fix a minimal parabolic subgroup Pmin of G(F), then the parabolic subgroups 
containing P,nin are called standard parabolic subgroups . Each parabolic subgroup 
in G(F) is conjugate to a standard parabolic subgroup. Let P be a parabolic 
subgroup of G{F). Among Zariski connected normal unipotent subgroups of P 
there is the maximal one. It is called the unipotent radical of P. Denote it by N . 
There exists a reductive subgroup M of P such that P = MA'' and M (1 N = {!}. 
Note that P is then a semidirect product of N and M. One says that P = MN is 
a Levi decomposition of P. Also, one says that M is a Levi factor of P. 



{tt{X)v,w) 



{v,tt{X)w) 
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Since a reductive group G{F) is a closed subgroup of GL{n,F), G{F) is in a 
natural way a locally compact group. If F is an archimedean field (i.e., = M or 
C), then G(F) is a Lie group in a natural way. If F is a nonarchimedean field, then 
G(F) is a totally disconnected group. 

In the sequel, a reductive group G{F) will usually be denoted simply by G. 

The most important examples of reductive groups are the classical groups such 
as general linear groups GL{n,F), special linear groups, symplectic groups and 
orthogonal groups. The groups GL{n,F) form the simplest series of reductive 
groups and one of the first series to be considered. 

Even if one is interested in harmonic analysis on some particular class of classical 
groups, it is usually necessary to study a broader class of groups because some 
important constructions involve subgroups which may not belong to the considered 
class. Such subgroups are reductive. This is the reason why it is convenient to 
use the terminology of reductive groups even if we consider some specific class of 
groups. 

Realizations of G. The set G has the following realizations, depending on whether 
the field is archimedean or not. We shall not use these realizations in the sequel, so 
the reader can also skip over these definitions. We note that the following notions 
are very important in the theory. It is also interesting to note how different G looks 
in the following archimedean and nonarchimedean realizations. 

Suppose that F is an archimedean field. First, wc shall give a definition of a 
(fl, iir)-module {3 is the Lie algebra of G). A representation n of a Lie algebra g is 
a real-linear map from q into the space of all linear operators on a complex vector 
space V such that 

7r([X,y]) = 7r(X)7r(r)-7r(y)7r(X) 

for any X,Y G q. If we have a Lie group G and a continuous representation tt of G 
on a finite-dimensional space V, then the following limit exists 

(L.A.) n{X)v = j^[TT{x{t)v)]t=o, vgV, 

and it defines a representation of the Lie algebra g of G on V (in the above formula 
X G Q and X is the tangent vector to the curve x{t) a.t t = 0). We call this Lie 

algebra representation the differential of tt. Suppose now that G is a reductive 
group over F and 6 the Lie algebra of K. Let (tt, V) be a pair where F is a 
complex vector space and tt = {ng^TTK) is again a pair consisting of a Lie algebra 
rcprcscintation TTg of g on F and of a representation ttk of K on V, such that the 
following three conditions arc satisfied. 

(a) For each v GV the vector space W spanned by all iTK{k)v, k € K, is finite 
dimensional and the representation of on W is continuous. 

(b) The differential of the representation ttk of K equals the restriction of the 
Lie algebra representation TTg to B. 

(c) For any k G K,X G g, and v gV 

Trs{Ad{k)X)v = 7^K(fc)7^s(X)7^if(fc-l)^;. 
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Then (tt, is called a {q, K)- module. An irreducible (g. K)-rnodule is a {q,K)- 
module which has no nontrivial subspaces invariant both for actions of K and g. 
Two (fl, ii')-modules {tt'jV) and {tt",V") are equivalent if there is a one-to-one 
linear mapping ip from V onto V" such that <^7r^(fc) = Tr'^{k)(p and <^7rg(X) = 
tt'^{X)(p for any k G K and X £ g. Now G is in a natural bijection with the set 
of all equivalence classes of irreducible (g, iir)-modules. If (7r,if) is an irreducible 
continuous representation of G (with finite Jf- multiplicities, which we always as- 
sume), then one takes for V the space of all ii'- finite vectors in H. The formula 
(L.A.) defines an action of g on V, and V becomes an irreducible (g, if )-module in 
this way. 

Suppose now that F is nonarchimedean. A representation (tt, V) of G is called 
smooth if for each v & V there is an open subgroup Ky of G such that Tr{k)v = v 
for any k G K^. Again we say that smooth representation (tt, V) is irreducible if 
there is no nontrivial vector subspace invariant for the action of G. Two smooth 
representations (711,^1) and (7r2, V2) of G are equivalent if there exists a one-to- 
one linear map (fi from Vi onto V2 such that W2{g)'fi = ^T^i{g) for any g G G. 
As before, there is a natural one-to-one correspondence from G onto the set of 
all equivalence classes of irreducible smooth representations of G. If (tt, H) is an 
irreducible continuous representation of G, one takes again for V the space of all 
i^'-finite vectors v in H. Now the restriction of the action of G on i? to ^ defines 
an irreducible smooth representation of G on V. 

3. Some simple constructions of unitary representations 

One would like to have rather simple and natural constructions of unitary repre- 
sentations which produce the whole of G. For nilpotent Lie groups such a systematic 
procedure consists of unitary induction by one-dimensional unitary representations. 
For the groups we consider, the situation is not so simple, but it is not too bad 
either. For example, one obtains the whole of SL(n, C)' by parabolic induction with 
one-dimensional, in general nonunitary, representations (see §9). 

We have introduced the topology of G in §1. In the construction of new uni- 
tary representations, the hardest problem is to find new connected components of 
G. Since G is not topologically homogeneous, there may exist special connected 
components, those consisting of only one point. These representations arc usually 
called isolated representations. To avoid the infiuence of the commutative harmonic 
analysis coming from G*^ = G/G'^^^ where G^'"' denotes the derived group of G, 
we shall define the notion of representations isolated modulo center. Let Z{G) be 
the center of G. For {tt,!!) e G there exists a character u)^ € Z{G)~ such that 
TT^z) = i^-niz) idij for all z € Z{G). The character is called the central character 
of TT. For a character a; e Z{G)~ set 

Go; = {tt e G; = w} and G^; = G^; n G. 

The representation tt g G (resp. n G G) will be called isolated modulo center (resp. 
isolated modulo center in the nonunitary dual) if {tt} is an open subset of Ga,„ 
(resp. an open subset of G^j^)- In the sequel, by isolated representation wo shall 
mean isolated modulo center. According to what we have said about the topology 
of G, we may say roughly that matrix coeflicients of isolated representations are 
not similar to other matrix coeSicients of elements of G. The following example 
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indicates that. If G is compact, then G is discrete, and matrix coefficients of 
different representations are L^-orthogonal. Kazhdan proved in [Ka] that the trivial 
representation is isolated when G is a simple group of split rank n > 2. The split 
rank is the highest n such that G possesses a Zariski closed subgroup defined over F 
which is isomorphic over F to GL(1, F)". As opposed to the trivial representation, 
other isolated representations are usually not easily constructible. In fact, isolated 
representations of G or G are very distinguished representations in known examples. 
Certainly, each isolated representation in the nonunitary dual, which is unitary, is 
also isolated in the unitary dual. 

The first isolated representations that one usually meets in the representation 
theory of reductive groups are square integrable. An irreducible unitary represen- 
tation (tt, H) of G is called square integrable modulo center if for any v,w G H, the 
function 

c/H^ \iTTig)v,w)\ 

is a square integrable fimction on G/Z{G) with respect to a Haar measure. We 
shall use the term square integrable instead of square integrable modulo center. 
Actually, the unitarity of some (tt, H) g G may be obtained from the above square 
integrability condition (note that the unitarity of the central character of tt must 
be assumed to be able to formulate the above square integrability condition). The 
square integrable representations are crucial for Plancherel measure and important 
for parametrizing the nonunitary dual. In known examples, they are very often 
isolated (in the unitary dual). 

The known examples show that the construction of a connected component, or 
at least a big part of it, reduces to the construction of isolated representations 
of reductive subgroups of G attached to parabolic subgroups and some standard 
simple and well-known constructions. Now we shall recall these standard simple 
constructions. The first and the oldest one is: 

(a) Unitary parabolic induction. Let P = MN be a Levi decomposition of a para- 
bolic subgroup P of G. For a continuous representation a of M, we have considered 
a also as a representation of P using the projection P = MN P/N = M. Then 
lndp{a) was called a parabolically induced representation of G. If (t is a unitary 
representation, then this process will be called unitary parabolic induction. In fact, 
we always take a G M. Then Indp(cr) is a unitary representation which is a direct 
sum of finitely many irreducible representations. It is usually irreducible. In the 
construction (a), we shall always assume that P is a proper subgroup of G. In 
general, if ( , ) is an M-invariant hermitian form on the representation space U 
of a, then 

(/l,/2)^ / {Mk)j2{k)) dfXKik) 

is a G-invariant hermitian form on Indp(CT), and it is positive definite if the form 
on U was positive definite. 

Unitary parabolic induction has been used since the first classification of the 
unitary duals of reductive groups [Bg], [GfNl]. Gelfand and Naimark started to use 
systematically unitary parabolic induction for the classical simple complex groups, 
while Harish-Chandra started a systematic study of this induction. 

The following construction was used also in the first classifications of unitary 
duals of reductive groups [Bg, GfNl]. 
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(b) Complementary series. It happens that some representations incinccd by nonuni- 
tary ones become unitary after a new inner product is introduced on the representa- 
tion space. The idea is the following. One realizes on the same space a "continuous" 
family (tTqjTJq), a € X, oi irreducible induced representations which possess G- 
invariant nontrivial hermitian forms. Let X be connected. Suppose that some Tr^ 
is unitary. The fact that a continuous family of nondegenerate hermitian forms on 
a finite-dimensional space parametrized by X, being positive definite at one point 
of X, must be positive definite everywhere enables one to conclude that all con- 
structed representations are in G (here one reduces arguments to finite-dimensional 
spaces by considering spaces where i5 runs over fixed finite subsets of K). 
Positivity at one point is obtained in general from (a). The delicate point is the 
construction of a continuous family of G-invariant hermitian forms, and it is based 
on the theory of intertwining operators for induced representations. 

For the above construction some authors use the term deformation [Vo4]. We 
have chosen rather a more traditional name. 

Let us recall that a topological space X is quasi-compact if each open covering 
of X contains a finite open subcovering. Note that in the above definition of a 
quasi-compact topological space, the Hausdorff property is not required (this is the 
difference between quasi-compactness and compactness). A topological space is lo- 
cally quasi-compact if each point has a fundamental set of neighborhoods which are 
quasi-compact. The fundamental fact about the topology of G (actually, about the 
dual of any G*-algebra) is local quasi-compactness. This fact essentially, together 
with some understanding of the topology of the unitary dual, implies that G cannot 
be complete without 

(c) Irreducible suhquotients of ends of complementary series. This fact was first 
observed and proved by Milicic. Before we give a brief argument why the rep- 
resentations in (c) must be included in G, we shall give a simple but suggestive 
example. 

Let P be a minimal parabolic subgroup in G = GL(2, F) (one may take for P the 
upper triangular matrices in G). We have denoted by Ap the modular character 
of P. Representations 

7" = Ind^(A^), -1/2 < a < 1/2 

are irreducible. If a = 0, then 7° is unitary by (a) (Ap is the trivial representation, 

so it is unitary). The family I", —1/2 < a. < 1/2, is a "continuous" family of 
irreducible representations with nondegenerate G-invariant hermitian forms. Thus, 
they belong to G by (b). 

We shall pay attention now to the representation at the end of these complemen- 
tary series = Indp(Ap^^^). From the definition of it follows that the 
trivial representation of G is a subrepresentation of 7~^/^ (the trivial representa- 
tion of some group G is a representation on a one-dimensional space V where each 
element acts as the identity on V). This is the unique proper nontrivial subrepre- 
sentation of 7~^/^. Since is infinite dimensional, there is no inner product on 
for which /^^/^ is a unitary representation (in a imitary representation for 
each subrepresentation there is another subrepresentation on the orthogonal com- 
plement). Nevertheless, the representation on the quotient of 7~^/^ by the trivial 
representation is unitary (actually, it is square integrable, and this implies that it 
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is unitary). So, though /"^/^ is not unitary, each irreducible subquoticnt of the 
representation at the end of the complementary series is unitary. This is the case 
in general. 

The representation /"^/^ from the above considerations is a representation which 
is not irreducible, but also, it is not very far from being irreducible. This is an exam- 
ple of a representation of finite length. Before we proceed further with explanation 
of the construction (c), we shall recall the definition of a representation of finite 
length. Suppose that we have a continuous representation (7r,if) of a reductive 
group G, which has finite ii'-multiplicities. Then we say that tt is of finite length if 
there exist subrepresentations 

{0} = HoCHiC...CHr, = H 

of H, such that the quotient representations of G on Hi/Hi-i are irreducible rep- 
resentations of G, for i = 1,2, ...,n. Parabolic induction carries the continuous 
representations of M of finite length to the continuous representations of G of 
finite length. 

Now we shall give a brief argument for the unitarity of representations in (c). 

We shall omit technical details. Suppose that we have a complementary series of 
representations tTc, a & X. We may consider the following situation. Let F be a 
topological space with a countable basis of open sets, and let X be a dense subset 
of Y. Assume that to each a eY is attached a nontrivial continuous representation 
TTc of G of finite length such that the functions 

are continuous, for all t/p from the space Gc,*{G) of all continuous compactly sup- 
ported functions which span a finite-dimensional space after translations by el- 
ements of K (left and right). Suppose also that the tt^ are irreducible unitary 
representations for all a G X. Take any a G Y. Let (a„) be a sequence in X 
converging to a. Milicic proved that in general a sequence (7r„) in G has no accu- 
mulation points if and only if lim„ 87r„ {^p) = for all (p [Mi, Corollary of Theorem 
6]. Since 0,r<, ^ 0, (tt^^) has a convergent subsequence. If S is the set of all limits 
of subsequences of (tTq,^) in G, then Milicic's description of the topology of G says 
that there exist positive integers no-, a E S, such that 

lim„ e^„^ ((yj) = ^ TlaQai^P) 

for all (f in Cc,*(G) [Mi, Theorems 6 and 7]. Also, 5 is a discrete and closed subset 
of G. Thus 

for all (f in Cc,*(G). Since the character of tt^ is the sum of characters of its 
irreducible subquotients, one can obtain easily that S is precisely the set of all 
irreducible subquotients of tTq. Since S C G, each irreducible subquotient of tTq is 
unitary. Thus (c) provides unitary classes. 

For a direct proof without use of the topology, one may consult [Td5] . The proof 
is based on the fact that a group of unitary operators on finite-dimensional Hilbert 
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space is finite dimensional. One uses in the proof the fact that reductive groups 
have large compact subgroups. 

While the constructions of (a) and (b) provide bigger continuous families of uni- 
tary representations, (c) provides smaller families, but they are often important in 
the construction of unitary representations. Representations obtained by construc- 
tions (a), (b), or (c) are never isolated. We shall describe now a simple construction 
found by Spch that may produce isolated representations. This construction is par- 
ticularly useful when it is combined with some other constructions, for example, 
with (a), (b), and (c). Contrary to previous constructions, here one gets unitarity of 
representations of smaller groups from unitarity of representations of bigger groups. 
Before we describe the construction, we need a notion of hermitian contragradient. 

For a continuous representation (tt, H) of G, 7f will denote the complex conjugate 
of TT. It is the same representation, but the Hilbert space is the complex conju- 
gate of H. The contragradient representation of tt will be denoted by tt. It is the 
representation on the space of all continuous linear forms on H with the action 
= ./("^(ff^^)^)- Set 7r+ = TT. Then tt"*" will be called hermitian contragra- 
dient of TT. A continuous irreducible representation tt will be called hermitian if tt 
and TT"*" are in the same class in G. It is easy to see that all unitary representations 
are hermitian. In the classifications of G it is usually easy to check whether tt is 
hermitian or not. 

Let P = MN be a proper parabolic subgroup of G. It is very easy to prove the 
following fact: 

(d) Unitary parabolic reduction. If we have hermitian a € M such that Indp(cr) is 
irreducible and that its class in G is unitarizable, then the class of a is unitarizable 
too. 

We shall now give a rough argument explaining why (d) provides unitarizable 
representations. Let {a, H) be an irreducible nonunitarizable hermitian represen- 
tation of M . Then there is a nondegenerate M-invariant hermitian form -0 on H. 
Now H decomposes H = H+(BH- as a representation of KCiP, where tp is positive 
definite on iJ+ and negative definite on Clearly, ^ {0} and ^ {0}. 
Note that Indp(c7) and lnd^f^p{a\K fl P) are isomorphic as representations of K 
and 

Indfnp(<T|i^nP) - Indfnp(i^+)©Indfnp(^^-)- 

Since unitary induction carries unitary representations to unitary, we see that the 
hermitian form induced by ■0 is indefinite. This form is also G-invariant. Since 
a G-invariant hermitian form on an irreducible representation is unique up to a 
scalar, we see that if Indp(f7) is irreducible, then it is not unitarizable. 

Roughly speaking, the construction (d) enables one to construct sometimes from 
a component of Mi a component of M2, where Pi = MiNi are two parabolic 
subgroups of G. 

There are also some simple constructions based on the geometry of a group or 
groups. For example, if cr, e Gj, then tri 0fT2 G (Gi x G2)' (and conversely). There 
are also irreducible unitary representations which appear already in the classifica- 
tion of the nonunitary dual — the square integrable ones. 

It is interesting to ask which constructions must be added to (a) (d) to gener- 
ate the whole unitary dual of the classical groups, starting with square integrable 
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representations. We shall see that for the first class, the case of GL{n,F), the 
constructions (a) -(d) are enough. 

Remarks. (1) Fell introduced in [Fe] a notion of nonunitary dual space for arbi- 
trary locally compact group. It is a topological space consisting of so-called linear 
system representations. We have studied G as a topological space in [Td6] if G is 
a reductive group over a nonarchimedcan field F. It was shown that G coincides 
with Fell's nonunitary dual. The set G is a closed subset of G. A representa- 
tion TT S G is isolated modulo center if and only if there is a nontrivial matrix 
coefficient which is compactly supported modulo center. Therefore, one may in- 
terpret Jacquet's subrepresentation theorem [Cs, Theorem 5.1.2] in the following 
way. Each element of G can be obtained as a subrepresentation of Indp(a-), with a 
isolated modulo center representation of M for some parabolic subgroup P — MN 
of G. Each isolated modulo center representation of G in G is essentially unitary 
(i.e., it becomes unitary after twisting by a suitable character of G); actually it is 
essentially square integrablc. Certainly, all these facts about the topology of the 
nonunitary dual should hold over archimedean fields with the proofs along the same 
lines. In the archimedean case, the representations with matrix coefficients com- 
pactly supported modulo center can exist only if G/Z{G) is compact. Construction 
of representations of such groups is essentially solved by the case of compact Lie 
groups. 

(2) Suppose that F is nonarchimedcan. Let P = MN be a parabolic subgroup 
in G. Let a € M he isolated modulo center. Denote by °G the set of all g € G 
such that the absolute value oi iJL{g) is one for any homomorphism fj, : G ^ F^ 
which is also a morphism of algebraic varieties defined over F. Then a character 
X : G — » is called unramified if x is trivial on °G, i.e., if xC^G) = {!}. Let U{M) 
be the set of all unramified characters of M. Then U{M)a = {xct; x £ U{M)} is a 
connected component of M containing a, and the set of all irreducible subquotients 
of Indp(T), T E U {M)<j, is a connected component of G. All connected components 
of G are obtained in this way [Td6]. So, for G the set of connected components 
reduces to the set of isolated representations modulo center in the nonunitary dual 
of the reductive parts of parabolic subgroups. Note that a difficult problem in the 
nonarchimedcan case is the construction of representations isolated modulo center 
(i.e., of supercuspidal representations). 

4. Completeness argument 

In the last section, we have outlined constructions (a)-(d) of unitary representa- 
tions of a reductive group G. It seems that those constructions provide a remarkable 
part of the unitary duals of the classical groups. This leads to one of the most in- 
teresting questions about unitary representations of reductive groups: How can one 
conclude that a set X C G is a significant piece of G, or even all of it? At the 
present time there is no satisfactory strategy for answering such density questions. 
Recall the simple answer for finite groups: One needs to check if the sum of squares 
of degrees of representations in X is equal to the order of the group or not. 

Suppose that a set X C G is constructed and suppose also that one expects 
that it is the whole unitary dual. If one wants to prove that, then a usual strategy 
has been to prove that in G\X all representations are nonunitarizable. One checks 
for each representation in G\X that it cannot be unitarizable considering various 
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properties of that representation. The simplest properties that one can consider 
are: if the representation is hermitian, if it has bounded matrix coefhcients, etc. 
The construction (d) can be used also for getting nonunitarity (from M to G). The 
above strategy we shall call the indirect strategy (of proving completeness of a given 
set of unitary representations). 

The indirect strategy becomes less satisfactory for groups of larger size. At the 
same time, the indirect strategy is not completely satisfactory from the point of 
view of harmonic analysis: the stress is not on unitary representations, which are of 
the principal interest, but on nonunitary ones. Actually, one needs a very detailed 
knowledge of the structure of representations in G\G, and the set G\G is usually 
much, much larger than the set G. The indirect strategy does not develop directly 
the intuition about unitary representations. In dealing with G, it is very useful 
to algebracize the situation (the algebraic description of G was presented in the 
appendix at the end of §2). 

Later on we shall present a completeness argument for GL(n), dealing simulta- 
neously with all GL(n) and having only one argument rather than various cases. 
This will be an example of the direct strategy of proving completeness. 

In the following section we shall explain the sense in which the set of represen- 
tations of Gelfand, Naimark, and Stein is "big" . 

5. On the completeness argument: the example of GL(n, C) 

We shall first introduce some general notation for the general linear group and 
then pass to the complex case. 

In the first part of this section, F denotes any local field. 
For X G , there exists a number |x|_f > such that 



for all / G Gc(G) {da,{g) denotes an additive invariant measure on F). Set \0\f = 0. 
Then | |j? is called the modulus of F. Note that | \r is the usual absolute value 
on R, I |c is the square of the usual absolute value on C (i.e., I^jc = zz), while 
|x|qp = \x\p (see the Introduction). For g e GL(n, F) set 



Clearly, v : GL(n, F) — > is a character. 

For ni,n2 € Z_|_, we have denoted by P(ni,n2) the parabolic subgroup of GL(ni -f- 
n2, F) consisting of the elements g = {gij) for which gij = when i > ni and j < ni. 
Also we have denoted M(^ni,n2) = {(ffu) ^ P(m,n2)'^9ij = for j > ni and i < 
m). Then 



is a Levi factor of P(„j n^). 

For two continuous representations Ti of GL(ni; F), i = 1, 2, we consider n ® T2 
as a representation of M(„j — GL(ni,F) x GL(n2,-F') in a natural way. The 
mapping mn ^ (n (8) r2)(m), where m e M(„j ^ ^ ^{ni,n2)^ defines a 

representation of This representation of P^^ni^n^) again denoted by 

n ®T2- Thus 




v{g) = |det(ff)|F. 



M(„^,„,)^GL(ni,F)xGL(n2,F) 



Ti (g) r2 : 



gi * 

52 



^ Ti (51) ® 7-2(52) 
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for gi G GL{ni,F) and 52 € GL(n2,-F). Now the parabolically induced represen- 
tation 

Indp ' (n <S^T'2) 

will be denoted by ti x T2. It is a standard fact that (n x r2) x ra is isomorphic to 
Ti X (r2 X Ts) (i.e., there exists a continuous intertwining which is invertible). This 
is a consequence of a general theorem on induction in stages 

Therefore, it makes sense to write n x r2 x T3. 

Before we explain an important property of the operation x , we need the notion 
of associate parabolic subgroups and associate representations. Suppose that we 
have two parabolic subgroups Pi and P2 of some reductive group G. If we have Levi 
decompositions Pi = MiNi, P2 = M2iV2, and w e G such that M2 = wMiur'^, 
then we say that Pi and P2 arc associate parabolic subgroups. Suppose that ai and 
a2 are finite length continuous representations of Mi and M2 respectively, such 
that cr2(?7i2) = ai{'w~^m2w) for all 7712 G M2. Then we say that ai and (72 are 
associate representations. For a continuous representation {tt,H) of G of finite 
length, consider a sequence of subrepresentations 

{0} = HoCHiC...CHk = H 

of H where the quotient representations on Hi/Hi-i are irreducible representations 
of G, for i = 1,2,..., k. Denote by R{G) a free Z-module which has for a basis 
G. We shall denote by J.H.(7r) the formal sum of all classes in G of the repre- 
sentations Hi/Hi-i, i = 1,2,..., A:. We shall consider J.H.(7r) € R{G) and call 
it the Jordan-Holder series of w. The element J.H.(7r) € R{G) does not depend 
on the filtration Hj , i = 0, 1, 2, . . . , fc as above (one can sec that from the linear 
independence of characters of representations in G). Suppose that Pi = MiNi 
and P2 = M2N2 are associate parabolic subgroups. Let ai and (T2 be associate 
representations of Mi and M2 (as before, we consider cri and (72 as representa- 
tions of Pi and P2 respectively) . A standard fact about parabolic induction from 
associate parabolic subgroups by associate representations is that representations 
Indp ((Ti) and Indp (o"2) have the same characters, which implies that these two 
representations have the same Jordan-Holder series. The formula for the character 
of a parabolically induced representation from a minimal parabolic subgroup, when 
= R, is computed in [Wr, Theorem 5.5.3.1]. A similar formula holds without 
assumption on parabolic subgroup. In the nonarchimedean case we have a similar 
situation. 

Suppose that ti and T2 are continuous representations of finite length of GL(n, F) 
and GL(m, F) respectively. The above fact about induction from associated para- 
bolic subgroups by associate representations implies that n x T2 and T2 x n have 
the same Jordan-Holder series. 

Set Irr" = U„>q GL(n, Py. To solve the unitarizability problem for the GL(n, F)- 
groups, one needs to determine Irr". 

In the rest of this section we shall assume F = C. Recall that | |c is the square 
of the standard absolute value that we usually consider on C. 

— 1/2 

Let xo : ^ be the character x x\x\f^ . Since each tt G GL(n, C)' has 
a central character and GL(n, C) is a product of the center and of SL(n, C), the 
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restriction of representations from GL(n, C) to SL(n, C) gives a one-to-one mapping 
of 

GL(n, C)\„ U GL(n, C)\2 U • • • U GL(n, C)"^^ 

onto SL(n, C)'. Therefore, in order to understand SL(n, C)" it is enough to under- 
stand GL(n,C)* (and conversely). In the rest of this paper wc shaU deal only with 
GL-groups and interpret the Gelfand, Naimark, and Stein representations in terms 
of GL(n). 

The first obvious irreducible unitary representations of GL(n, C)" are one-dimensional 
representations x ° det„ where x is a unitary character of and where det„ de- 
notes the determinant homomorphism of GL(n). Gelfand and Naimark obtained 
also the following series of irreducible unitary representations 

(z.-«x) X {u^x) = W'^ix ° det)] X K(x o det)], x e (CX)^ < a < i, 

the complementary scries representations for GL(2, C). These complementary series 
start from representations 

X X X = (xodet) X (xodct), x^^CC^Y- 

The following unitary representations of GL(n, C) will be obtained by parabolic 
induction using the above representations. Gelfand and Naimark showed in [GfN2] 
that the unitary representations obtained by parabolic indiiction using the rep- 
resentations above are irreducible. They assumed that in this way one gets all 
irreducible unitary representations of GL(n, C), up to unitary equivalence. 
Wc can interpret the above remarks in the following way. Set 

Bo = {xodet, K"(xodet)] X [i/"(xodet)]; xe(C^r, n e N, < a < i}. 

nil ^ 

Gelfand and Naimark showed that for ti , . . . , Tfe G _Bo , the representations ri x • • ■ x 
Tfc are in Irr" . Their assumption was that in this way one can get any representation 
from Irr". 

Stein showed that the Gelfand and Naimark complementary series representa- 
tions for GL(2, C) are just the first of the complementary series representations 
which exist for all GL(2n, C) [St]. He showed that 

[zv-«(x o det)] X [i/"(x o det)] G Irr" 

n n 

also for n > 2, < a < 1/2, x G (C^)', and he showed that these representations 
were not obtained by Gelfand and Naimark. These complementary series start from 
representations 

(xodet) X (xodet), n>2, xe(C^)^ 

n 71 

which were already well known to Gelfand and Naimark. 

Now it is natural to complete the Gelfand and Naimark list by the Stein com- 
plementary series representations. Therefore, put 

B = {xo det, [iy-"{x o det)] x [i/"(x o det)]: x e (C'')^ n G N, < a < i} 

n n n 

(i.e., B is just Bq completed by the Stein complementary series representations). 
Now using arguments similar to those of Gelfand and Naimark, one may conclude 
that for Ti, . . . , Tj, G -B the representations ti x ■ ■ ■ x are in Irr" (sec [Sh]). Let 
us denote by {G.N.S.} the set of all representations obtainable in this way. 

We can explain now in what sense {G.N.S.} is big in Irr". It is easy to prove the 
following fact (and we shall prove it later): 
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(D) For any tt e Irr", there exist ti, T2 G {G.N.S.} such that tt x n and T2 have 

a composition factor in common. 

It is clear that (D) plays a role in the completeness argument. 
Regarding unitary parabolic induction for GL(n, C), the simplest situation would 
be if it wore always irreducible. This is what Gelfand and Naimark expected to 

hold in 1950: 

(51) Unitary parabolic induction for GL{n,C) is irreducible, i.e., 

Ti , T2 e Irr" implies n x T2 e Irr" . 

Let us suppose that (SI) holds. Because of (SI) and (D), for each tt e Irr" there 
exist Ti, T2 e {G.N.S.} such that tt x n = T2. Thus, to obtain Irr", it is enough to 
know how representations from {G.N.S.} can be parabolically induced. 

We shall call n € G primitive if there is no proper parabolic subgroup P = MN 
and cr G M so that n = Indp(cr). Certainly, each tt G G is unitarily equivalent to 
some lndp{a) where G M is primitive. Wc have mentioned that if Pi and P2 arc 
associate parabolic subgroups and ai, associate representations, then Indp^(<Ji) 
and Indp^(i72) have the same Jordan-Holder series. The simplest situation would 
be if the converse were true for a\ and 02 primitive (because of the induction in 
stages, it is necessary to assume that u\ and are primitive). For GL(n, C) this 
would mean (having (SI) in mind): 

(52) If both families Ti G GL(ni,C)', i = l,...,n, and aj G GL(mj,C)",j 
1 II: . consist of primitive representations, and if 

n X • • • X T„ = CTl X • • • X am, 

then m = n, and after a renumeration, the sequences (ti,... ,t„) and (cti,... 
. . . ,(7to) are equal {all Ui, mj are assumed to be > 1). 

We shall assume that this holds. 
A very plausible hypothesis is: 

(53) The Stein representations 

K"(xodet)] X [i."(xodet)]; xG(C^)', n G N, < a < ^, 
are primitive. 

Certainly, if we assume (S3), then all elements of B are primitive. Now it is 
obvious that (D), together with (SI), (S2), and (S3), implies 

Irr" = {G.N.S.}. 

It remains to find a way to prove (SI), (S2), and (S3). Note that until now only 
(classes of) unitary representations were necessary, and statements (SI), (S2), and 
(S3) are essentially analytic. So, if one could prove (D), (SI), (S2), and (S3) dealing 
only with unitary representations, one would have a classification of GL(n, C)" 
completely in terms of unitary representations. 
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As wc shall sec, (D) is easy to prove using noncomplicated parts of the nonunitary 
theory. The following strategy for proving (S2) and (S3) simultaneously can be 
used. The set Irr" can be embedded in a suitable ring which is factorial and 
where multiplication corresponds to parabolic induction. Then one can prove that 
elements of B are prime or close to being prime. This would imply (S2) and (S3). 
This strategy would also include nonunitary theory (but again, not the complicated 
parts). 

Finally, we leave the discussion about (SI) for §9. Let us say that the first ideas 
for proving (SI) are due to Gelfand, Naimark, and Kirillov. 

In the following sections we shall elaborate in more detail the above strategy and 
outline such a strategy for GL(n) over general local field F. 

6. The nonunitary dual of GL{n,F) 

In this section, we state some basic facts about a paramctrization of GL(n, i^)~. 
Besides Irr" which was introduced in the previous section, we introduce 

Irr= y GL(n,F)'". 

n>0 

The set of all classes of square intcgrable representations in Irr" of all GL(n, F),n > 
1, will be denoted by £>". The set of all essentially square integrable representations 
will be denoted by D. More precisely, 

D = {{xodet)5;x&GL{l,Fr, SeD^}. 

For a set X, M{X) will denote the set of all finite multisets in X. These arc all 
unordered n-tuples, n € Z+. This is an additive semigroup for the operation 

(ai, . . . ,a„) + (6i, . . . ,6m) = (ai, . . . , a„, 6i, . . . , 6^). 

By Wp we shall denote the Weil group of if F is archimedean and the Weil- 
Deligne group in the nonarchimedean case. For the purposes of this paper, it 
will not be essential to know exactly the definition of Wp- We denote by I the 
set of all classes of irreducible finite-dimensional representations of Wp- By the 
local Langlands conjecture for GL(n) (which generalizes local class field theory), 
there should exist a natural one-to-one mapping of Irr onto classes of semisimple 
representations of Wp, i.e., onto M{I) 

Irr^M(/). 

Under such a mapping, D should correspond to /. Thus, there should exist a 
parametrization of Irr by M{D). Let us write one such parametrization. 
Let a = {5i, . . . , (5„) e M{D). We can write 

5i = v<^<-'^b'l, e(<5i) e K, <5," e D". 

After a renumeration, we may assume e{8\) > €{62) > ■ • ■ > e(5„). The represen- 
tation 

A(a) = 61 X ■ ■ ■ X Sn 
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has a unique irreducible quotient (possibly A(a) itself), whose class depends only 
on a [BlWh, Jcl]. Its multiplicity in A(a) is 1. We shall denote this class by L{a). 
Now 

a H^. L{a), M{D) Irr 

is a one-to-one mapping onto Irr. It is a version of the Langlands parametrization 
of the nonunitary duals of GL(n)-groups. Certainly, for the existence of such a 
parametrization, it is crucial that the parabolic induction by square integrable 
representations is irreducible for GL(n). 

The formula for the hermitian contragradient in the Langlands classification 
becomes 

L((5i,...,<5„))+ = L((5+,...,<5+))- 
If we write Si = 5f {e{Si) G R, 5V; G D"), then 

Also, it is easy to show that 

...,Sn)) = L{{u''5u a e C. 

Let Rn be the free abelian group with basis GL(n,F)'", i.e., i?„ = R{GL{n,F)). 
For each finite- length continuous representation tt of GL(r!, F), we have denoted by 
J.H.(7r) its Jordan-Holder series which is an element of Rn- Set 

R = ^Rn- 
n>0 

Now ii is a graded additive group which is free over Irr. Let 

Rn ^ Rm ^ Rn+m 

be the Z-bilinear mapping defined on the basis Irr by 

(ij,t) !->■ J.H.(cr X r). 

This defines a multiplication on R, which will be denoted again by x. We have 

mentioned in the previous section that r x cr and cr x t have the same Jordan- 
Holder sequences. This just means the commutativity of R. Since (ti x T2) x T3 = 
Ti X (t2 X T3) (§5), R is also associative. Certainly, {L{a); a G M{D)} is a 
basis of R. It is a standard fact that {A(a); a G M{D)} is a basis of R (i.e., 
standard characters form a basis of the group of all virtual characters). This means 
nothing else than the following fact which was first noticed by Zelevinsky in the 
nonarchimedean case [Ze, Corollary 7.5]. 

6.1. Proposition. The ring R is a Z-polynomial algebra over all essentially square 
integrable representations {i.e., over D). 

In particular, i? is a factorial ring. 

It is a natural question to ask if it is possible to relate the operation of summing 
representations of Wp with some operation on representations from Irr in the cor- 
respondence a H- > L{a); i.e., is there a relation between L{a + b) and representations 
L(a), L{b)l The answer is very nice:L(a + 6) is always a subquotient of L{a) x L{b). 

One may find more information about the Langlands classification on the level of 
general reductive groups in [BlWh] . For the Langlands philosophy one may consult 
[Gb3]. 
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7. Heuristic construction 

In this section we shall try to see what would be the part of the unitary dual for 

the GL(n)-groups over arbitrary local field F, generated by classical constructions 
(a)-(d) of §3. Our principle will be to expect a situation as simple as we could 
assume, bearing our evidence in mind. 

So, let us start with D". The representation (5 x (5 is irreducible, so S x S Cz Irr" 
by the construction (a). Then we have the complementary series which starts from 
SxS: 

[ly^S] X [i^-^S], < a < 1/2. 

This is an example of construction (b). We require a < 1/2 to ensure that the 
induced representation is irreducible. At the end of complementary series [p^S] x 
[z/"'*^], there will be unitary irreducible subquotients. To be able to identify at 
least one, let us recall the relation mentioned at the end of §6; namely, 

L{a + b) is a subquotient of L{a) x L{b) for a,b e M{D) . 

We shall assume that this holds in the rest of this section. Therefore, we have 
that L{{i^^/^S,u-^/^ (5)) is unitary since it is at the end of the above complementary 
series (construction (c)). 

To proceed further, in order to be able to form a new complementary series, let 
us suppose that for general linear groups unitary parabolic induction is irreducible. 
Then one has I/((:^^/^(5, i/~^/^i5)) x ^((z/^/^J, e Irr", and further, one has 

a new complementary series 

[l/«i((z/V2j^ 2,-1/2^))] X [z/-°i((z/V2j^ 2,-1/2^))]^ < a < 1/2. 

At the end of this complementary series 

= L{{i^5,6)) X 
is the unitary subquotient 

L{{iyS, 5,v'^6, S)) 

by the assumption that L{a + 6) is a subquotient of L{a) x L{b). It is natural to 
ask if the above representation is primitive. But L{{i'5, S, u^^S, S)) is a subquotient 
of 

L{{i^6J.u-^S)) X LiiS)). 

Note that L{{iy'6, S, i'~^S)) (g) L{{6)) is hermitian. If we take the simplest possibility, 
that is, L{{u6,d,i'~^6)) x L{{5)) irreducible, then the construction (d) implies that 

L{{uS, 6, i^-^S))®L{{S)) 

is unitary. Clearly then, L{{iyS,S,iy~^S)) € Irr". 

At this point it is convenient to introduce some notation. Set 



a(7, n) = . . . , u-^^-^^^j) 
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for 7 G £) and n G Z+ and 

u(7,n) = L(a(7,n)). 

We shall write 

Now we proceed further. We already have u{5,l), u{5,2), u(5, 3) G Irr". One 
considers a complementary scries 

[j/"m((5,3)] X [j/-"w((5,3)], 0<a<l/2, 

which starts from u{5, 3) x u{5, 3). At the end we have the representation 

L{v^'^a{5,i)) X L{v-^'^a{5,2,)), 

and we can identify one irreducible subquotient which is 

L{v^l^a{5,i) + v-^l^a{5,i)). 

It is a unitary subquotient by (c). Note that 

v^''^a{5, 3) + v-^''^a{5, 3) = a{5, 4) + a{5, 2) 

which means that L(a(5, 4) + a{5,2)) is unitary. Further, this representation is a 
subquotient of 

u{5,A) X u{5,2) = L{a{5,4:)) x L{a{S,2)). 

Suppose again that u{S, 4) x u{S, 2) is irreducible. Since it is unitary, u{S, 4)(E)u{S, 2) 
needs to be unitary by the construction (d). Therefore, u{6,4) will be unitary. 
Now it is easy to conclude that the assumption 

u{6, n) X u{6, n - 2) G Irr 

leads to 

u{5, n) G Irr" . 

In the case of GL(n,C), D" is equal to GL(1,C)^ = (C^)'. Then u{5,n) = 
5 o det„. So, we have not obtained new unitary representations in this case. 

8. Scheme of unitarity for GL(n) 

In the last section we have seen heuristically what some simple assumptions sug- 
gest. Now we shall write down some of those assumptions and their "implications". 
We first recall that 

u{5, n) = L{a{6, n)) = L((i/("-i)/2^, iy("-3)/2(5, . . . , zy-(»-i)/2^)) 

for S G D and n > 1. It was also observed in §6 that is a factorial ring. 
We introduce the following statements: 

(UO) r, cT G Irr" r X a G Irr. 

(Ul) (5 G D", n G N ^ u{S,n) G Irr" . 

(U2) (5 G D", n G N, < a < 1/2 ^ [i/"u((5,n)] x [u-°'u{6,n)] G Irr" . 

(U3) 6 € D, n G N u{6, n) is a prime element of R. 

(U4) a, 6 G M{D) =^ L{a + 6) is a composition factor of L{a) x L{b). 
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Here only (U3) was not assumed or obtained in the last section. It is a strength- 
ening of the assumption that the n)'s are primitive, which was present in the 
last section (otherwise, we would have tried to construct new unitary representa- 
tions in that way). 

We have seen that (UO) and (U4) lead to (Ul) and (U2) (i.e., unitarity of the 
representations mentioned there). But it is interesting and surprising that with 
the addition of only one assumption, namely, (U3), the preceding assumptions also 
easily imply completeness for the unitary duals of the groups GL(n, F). 

8.1. Proposition. Suppose that (U0)-(U4) hold true. Set 

B = {u{6,n), [u'^u{S,n)]x[i^-°'u{5,n)], ^ G n e N, < a < i}. 
Then: 

(i) if Ti, . . . ,Tk G B, we have n x • • • x rfe G Irr"; 

(ii) i/ TT G Irr", then there exist a\, . . . ,<Jm & B such that 

TT = (Ti X • • • X (T„; 

(iii) if ai, . . . ,ak, ti, . . . ,Tm & B and ai x ■ ■ ■ x ak = ti x ■ ■ ■ x Tm, then k = m 
and the sequences tri, . . . ,ak and n, . . . , coincide after a renumeration. 

From (UO), (Ul), and (U2) one obtains (i) directly. Also (U3) imphes (iii). It 
remains to prove only (ii). First we shall prove 

8.2. Lemma. Suppose that tt g Irr is hermitian. If (U4) holds, then there exist 
(Ti, . . . , (T„, Ti, . . . , Tm G i? such that TT X <Ti X • • • X ij„ and ri x • • • x have a 

composition factor in common. 

Proof Let (5 G fc G (1/2) Z+, and < /3 < 1/2. Then 

(!/-'= 5, ly'^S) + a{d, 2k -1) = a{6, 2k -hi), k> 0, 

and 

{u-'^-l^d, i^'^+^^S) + iy^^-^/^a{5, 2k) + v^^'^-^a{b, 2k) 
= iy^^a{S, 2k + 1) + i^-^a{6, 2k + 1). 

Let TT G Irr be hermitian. Then tt = i((7i, . . . , 7s)) for some 7i G D. Now 

L((7i,...,7«))+ = i((7i+,...,7+)) 
(see §6). This implies that we can write tt in the form 

("1 "2 "3 

E(^^"''<5i,^^'''5^)+ £ (i^^'^'-'^'^i, i/'=-+'^'5i) + ^ ('^») 
i=l i=ni + l i=n2 + l 

where Si G fc, G (1/2) Z+, < < 1/2 and all fci,... ,A;„i > 0. Here 

ni,n2,n3 G Z+ and n2 > ni,ns > n2 (i.e., not all three sums need to show up 
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in the above formula). The two relations at the beginning of the proof and (U4) 
imply that both 



TT X 



Y[u{Si,2ki - 1) 



Li=l 



i=ni+l 



and 



'[[u{5i,2ki + l) 



.1=1 



f] {uf^'uiSi, 2ki + 1) X u-^'u{5i, 2ki + 1)) 



i—ni-\-l 



n 

.1=712+1 



have 



Lij^a{5i,2ki + 1) 
\i=i 

"2 "3 N 

+ 2 {u^^a{6i,2ki + l) + u-'^^a{5i,2ki + l))+ ^ {6i) 



i=ni + l 



i=n2 + l 



as a composition factor. This completes the proof of the lemma. □ 

End of proof of Proposition 8.1. Let tt G Irr". Then, by the preceding lemma there 
exist ai, tj G B so that tt x cti x • • • x cr„ and n x • • • x have a composition factor 
in common. Since (UO), (Ul), and (U2) imply that both sides are irreducible, we 
have 

TT X (Tl X • • • X Cr„ = Tl X • • • X Tm . 

Since R is factorial and the n)'s are prime by (U3), tt is a product of some 
u{S,kys, S € D. But the fact that tt is hermitian implies that tt is actually a 
product of elements of B. So, we have proved (ii). □ 

8.3. Remark. For GL(n) over a central simple division F-algebra, we expect that 
a scheme of this type should work too. In that case a slight modification in the 
definitions of the u((5, n)'s and lengths of complementary series is necessary (see 
[Td7]). 



9. On proofs 



We have seen that the fulfillment of (UO) (U4) implies a complete solution of 
the unitarizability problem for GL(n, _F), as was described in Proposition 8.1. 

Let us remark that (U0)-(U4) were expected to hold for F = C (except maybe 
(U3) because such questions were not considered). Statements (Ul) and (U2) were 
known, (UO) was expcx-tcxl even by Gelfand and Naimark, while (U4) is easy to 
prove. A simple consequence of (U0)-(U4), namely, the description of the unitary 
dual of GL(n,C), was not generally expected to hold. Now we shall make a few 
remarks on the history and proofs of (U0)-(U4). 
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Wc start with the statement (U4), which belongs to the theory of the nonunitary 
dual. This fact was proved by Zelevinsky in the case of nonarchimedean F for his 
classification of GL(n, F). His proof uses induction on Gelfand-Kazhdan derivatives 
[Ze, Proposition 8.4]. Rodier noticed in [Ro] that Zelevinsky's proof implies (U4) 
for the Langlands classification in the nonarchimedean case. We proved (U4) in a 
simple manner for the archimedean case [Td2, Proposition 3.5. and 5.6]. Such a 
proof is outlined for nonarchimedean F in Remark A.12(iii) of [Td3]. Sometimes 
one can conclude the equality in (U4), using the Zelevinsky's proof of Proposition 
8.5 in [Ze]. 

9.1. Proposition. Let a, = (JJ , . . . , J^J £ M{D), i = 1,2. If 6l x S"^ € Irr for 
all 1 < k <ni and 1 < m < n2, then 

L{ai) X L{a2) = L{ai + 02). 

The above proposition may be helpful in constructing complementary series. 

Let us now consider (Ul). Certainly if F = C, then there is nothing to prove 
since as we already mentioned, £>" = (C^ Y and for x G D", m(x, n) = x° dct„. 

For F = R, £)" C GL(1,IR)' U GL(2,R)^ Again, (Ul) is evident if x £ (K'')^ 
Speh considered the remaining case of u{S,n), 5 G GL(2,K)' n £>" [Sp2]. She 
proved unitarity using adelic methods. Surprisingly, it seems that Gclfand and 
Graev were already aware of this series of representations in the 1950s (see [Sp2, 
Remark 1.2.2.] about [GfGr]). 

For nonarchimedean F, we have determined in [TdS] the representations u{S, n) 
through the ideas presented in §7. Unitarizability is proved there essentially along 
those lines. It is also possible to prove unitarizability by the method of Speh as was 
done in the appendix of [TdS]. Note that here n GL(n, F)" ^ for all n > 1. 

For F = C, (U2) was proved by Stein in [St]. In general, (U2) follows from (UO), 
using the irreducibility of representations u"u{S,n) x i'~°'u{S,n),0 < a < 1/2, 
obtained from Proposition 9.1 and from the analytic properties of intertwining 
operators. There is also another method in the nonarchimedean case presented in 
[Bn2]. 

To prove (U3), one considers the m((5, n)'s as polynomials and proves the irre- 
ducibility of these polynomials. Here one uses the fact that i? is a graded ring and 
that w((5, n)'s are homogeneous elements. In the proof, one uses basic facts about 

the composition scries of generalized principal series representations (one does not 
need more detailed information, such as that obtained from Kazhdan-Lusztig type 
multiplicity formulas). It is a bit surprising that, although we do not know how to 
write down the polynomials u{5,n), wc can nevertheless carry out the proof. For 
proofs of (U3) see [TdS] and [Td2]. The statement (US) is obvious for u{5,l) by 
Proposition 6.1. 

Finally, let us return to (UO). Let P„ denote the subgroup of GL(n,F) of all 
matrices with bottom row equal to (0, ...,0, 1). Already Gelfand and Naimark 
noticed the importance of the statement 

(I) If TT e GL(n, FY, then 7r|P„ is irreducible (n e N). 

Actually, they proved the above statenic^nt for F ~ C, for the representations 
that they expected to exhaust the unitary dual of GL(n, C). Several people were 
aware that the above statement implies (UO). For a written proof see [Shj. Proof 
of the implication is based on Mackey theory and Gelfand-Naimark models. 
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Kirillov stated (I) as a theorem in [Kil] for F an archimedcan field. There he 
sketched a proof. Vahutinskii's classification of representations of GL(3, M)" was 
based on the proof. Having in mind a correspondence obtained by Mackey theory 

Pn ^ GL{n - 1, FY U GL(n - 2, F)' U • • • 
U GL(2, Py U GL(1, Py U GL(0, F)^ 

the statement (I) would imply that one would have simpler realizations for rep- 
resentations from GL{n,Py. It is from this setup that the name Kirillov model 
appears. In [Kil] Kirillov's intention was to prove that 7r|P„ is operator irreducible 
(i.e., the commutator consists only of scalars), which is enough by Schur's lemma 
to see the irreducibility of 7r|P„. One takes any T from the commutator of 7r|P„ 
and considers a distribution 

At : Trace (r7r(^)) 

on Gh{n,P) which is invariant for conjugations with elements from P„, since T 
is P„-intertwining. Now if At is GL(n, P)-invariant, then using the irreducibil- 
ity of TT, it is not difficult to obtain that T must be a scalar. Kirillov indicated 
that he proved in [Kil] that At is GL(n, P)-invariant (however, see below). This 
property of the distribution is especially easy to see when tt is a continuous finite- 
dimensional representation. Then the distribution At is given by a continuous 
function which must be constant on P„-conjugacy classes (At is P„-invariant). 
Since in GL(n, P), GL(n, P)-conjugacy classes contain dense P„-conjugacy classes, 
At must be GL(n, P)-invariant. 

Bernstein proved in [Bn2] that for P nonarchimedean, each P„-invariant distri- 
bution is GL(n, P)-invariant. He proved (I) using essentially the Kirillov's strategy. 
Besides proving (UO) in the nonarchimedean case, he gave a different proof of the 
implication (I) (UO). 

Bernstein states in [Bn2] that Kirillov's proof of (I) for F archimedean in [Kil] is 
incorrect, and he wrote that he himself had an almost complete proof (see [Bn2, p. 
55]). In any case, there is no written complete proof of (I) in the archimedean case 
now. We would say that Kirillov's proof is incomplete rather than incorrect. He 
failed to give a complete argument that the distribution At is GL(n, F)-invariant. 
As was noted by Bernstein, the tools used in Kirillov's paper do not seem to be 
sufficient for proving (I). The distribution At is a very special one. Actually (I) 
would imply that it is a multiple of an irreducible character; so by Harish-Chandra's 
regularity theorem, it is locally and analytic on regular semisimple elements. 
So if one proves that the (eigen) distribution At is locally and analytic on 
regular semisimple elements, one could apply finite-dimensional argument. This 
may provide a strategy to prove (I) in the archimedean case. This would be a 
longer proof, and there are also some disadvantages in proving (UO) through (I). 
We shall say a few words about these disadvantages. Before that, observe that 
there is an implicit proof of (UO) in [Vo3] . 

We have mentioned that the approach to the unitary dual of GL (n.F) is ex- 
pected to be applicable to GL(n) over a central division P- algebra A. As far as 
we understand, Vogan's description of the unitary dual of GL(n, H) confirms this. 
Here (UO) cannot be proved through (I), simply because (I) is false in general in 
this case. The simplest example can be obtained for GL(2,j4) {A ^ F). It is not 
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difficult to sec tliat tliere exist (irreducible) tempered representations which are 
reducible when restricted to a nontrivial parabolic subgroup. Thus, it may be more 
reasonable to search for proof of (UO) which works also for division algebras. There 
are some candidates for it (sec the last remark at the end of this section) . 
After all, we have the following: 

Theorem. Let F be a locally compact nondiscrete field. Set 

B = {u{6,n), [u'^u{S,n)]x[iy-°'u{d,n)], 6 € D"", n G N, < a < i}. 

Then 

(i) i/ (Ti, . . . , (Jfc G B, then CTi x • • • X crfe € Irr"; 

(ii) if n & Irr", then there exist ai,. . . ,am & B, unique up to a permutation, 
such that 

TT = (Ti X • • • X am ■ 

We remind the reader once more that there is no written complete proof yet 
of (UO) in the archimedean case, but there is a complete proof [Vo3] of the above 
theorem in this case. 

Remarks. (1) We give in [Td4] a concrete realization of the topological space 
GL(n, F)" when F is nonarchimcdean. 

(2) The last theorem, together with [Ka], implies that tt € SL(n, C)' is isolated 
if and only if tt is the trivial representation and n^2. 

(3) Let F be nonarchimcdean. Let p G Irr" be a representation having a non- 
trivial compactly supported modulo center matrix coefficient. Then for A; G N, the 
representation 

j,('=-l)pX!.('=-3)pX...XI.-('=-l)p 

has a unique square intcgrable subquoticnt which will be denoted by 5{p,k). In 
this way one obtains all £>" (see [Ze] and [Jcl]). In [Td4] we have proved that 
TT G GL(n,F)" is isolated modulo center if and only if tt equals some u{6{p, k),m) 
with fc ^ 2 and m ^ 2. 

(4) One could try to prove (UO) for GL(n) over local division algebras by proving 
first the following conjecture: Let A be a central local simple algebra, let S be the 
subgroup of the diagonal matrices in GL(2,A), let N be the subgroup of upper 
triangular unipotent elements in GL(2,A), and let a be an irreducible unitary 
representation of S. Then Ind^jy ' (u) should be irreducible. 

(5) A proof of (U3) and (U4) for GL(n) over a local nonarchimedean division 
algebra is contained in [Td7] . 
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Let us explain our classification of tlie references into several groups. The clas- 
sification is not very rigid. To each reference we have attached a superscript that 
indicates the group where it belongs. Here is a description of the groups. 

sur denotes the group of survey and expository papers, 
gen denotes the group of general references. 
GL (n) denotes the group of papers that are directly related to the topic of our 
paper. They are very useful for the further understanding of the topics 
discussed in our paper. The complete proofs that are omitted in this paper 
can be found in the papers from this group, 
his denotes the group of the historically important references for the topic of 
this paper. 

Lan denotes the group of references directly related to the Langlands program 
and groups GL(n). These references are very often related to the various 
correspondences predicted by the Langlands program for groups GL(n). 
Generally, these references are directed to more demanding and experienced 
readers. 

oth denotes the group of remaining useful references. They are mostly related 
to unitarity. 
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